00100	During the past several years our  vision  effort  has  shifted  from
00200	simple scenes of polyhedra to complex objects in richer context.  Not
00300	all problems with towers of polyhedra have been solved, but there are
00400	a  host  of  reasons  for moving on.  Manipulation has also proceeded
00500	from manipulation of blocks to assembly operations for  many  of  the
00600	same  reasons.    Minsky and Papert have described the central issues
00700	and the intellectual position of robotics in artificial intelligence.
00800	We refer to that discussion as background to our motivation to direct
00900	research to real world scenes and objects.  An  important  motivation
01000	is  to  develop  applications  for  manipulation  and visual feedback
01100	capabilities which have been developed here.  The time is  ready  for
01200	the   first  transfer  of  robotics  technology  from  laboratory  to
01300	prototype assembly stations. We discuss  robotics  applications  more
01400	fully in another section.
01500	
01600	Another  major  motivation is to face new issues; systems have come a
01700	few  steps  toward  heterarchical  systems  from  heirarchy;  from  a
01800	succession  of homogeneous passes with compounding errors to a system
01900	structure where high and low levels interact closely.  One aspect  of
02000	such a system is the use of many functionally independent techniques; this
02100	has meant including experimentation with  color  and  texture,  depth
02200	information,  motion  parallax  and  image prediction.  The choice of
02300	problems as paradigms is central  to  the  way  in  which  perceptual
02400	systems  evolve.    A  great  deal  can be done with simple scenes of
02500	polyhedra using a single image  in  one  color,  ignoring  all  other
02600	perceptual abilities.  Since most perceptual problems can be resolved
02700	within  a  single  image,  one   approach   is   to   exhaust   those
02800	possibilities.    Important  areas  are  missed;  for example, visual
02900	feedback is clumsy without depth information.   For our plans, visual
03000	feedback for manipulation is primary.
03100	
03200	There  are  other  ways  in  which  the  tower  of blocks paradigm is
03300	inadequate  for  our  purposes.    Low  level   feature   description
03400	operators   are   tailored   for   this  case.  Adequate  region  and
03500	edge-oriented descriptors must  be  evolved.   With  polyhedra,  only
03600	simple  preliminary  grouping  mechanisms have been programmed.  More
03700	intermediate level descriptors must be implemented.
03800	
03900	Although real world scenes are much more complicated, they  are  also
04000	simpler  in that they have a much richer context.  There is only very
04100	general knowledge available in the usual blocks task; usually  color,
04200	size,  and  support  are unspecified.  In an assembly task, sizes are
04300	known, color is used to code, and parts fit in special  ways.   Often
04400	recognition  can  be  based on obvious special features.  These tasks
04500	are good exercise in developing goal-oriented visual  systems.  To  a
04600	great  extent, a field is determined by its paradigms: thus character
04700	recognition has been a weak paradigm in  that  almost  any  technique
04800	works  for limited fonts, and many important perceptual abilities are
04900	unnecessary.   Techniques  developed  from  that  domain   have   not
05000	contributed much to three-dimensional vision.
05100	
05200	Techniques  for vision in the blocks world are relatively mature; the
05300	conceptual payoff has  been  substantial,  particularly  in  symbolic
05400	description  of  perfect line drawings, as exemplified by the work of
05500	Huffman, Clowes, and Waltz.  We continue an interest  in  the  blocks
05600	world as a vehicle for strategies and perceptual problem solving.
     

00100	overall significance of information sciences in industry
00200		batch production
00300		control of scheduling, resource allocation, flow routing
00400		commonality of design
00500			find common fabrication and assembly operations
00600			find more common components
00700		increased understandibility and communication
00800			availibility of information
00900				keep track of delayed effects of design decisions
01000				cuts cost of preparation and transmission
01100				increased accessibility of information
01200		
01300	
01400	Although development  and  utilization  of  intelligent  robots  will
01500	progress  gradually,  the  information  sciences will impact industry
01600	enormously over a shorter term. An immediate effect is to  streamline
01700	information  flow  in the manufacturing process.   Simple systems can
01800	keep track of delayed effects of  design  decisions,  lower  cost  of
01900	preparation  and  transmission,  and increase accessibility. Computer
02000	controlled  scheduling,  resource  allocation,   and   transportation
02100	routing  are  important  to  most  of industrial production [Anderson
02200	1972]. We see a major contribution from techniques being pioneered in
02300	artificial  intelligence  laboratories,  and  that such research will
02400	have payoff throughout the entire development of intelligent systems.
02500	Enormous efficiency increases will come from analysis of assembly and
02600	inspection in terms of some "primitive  operations".   In  the  past,
02700	computer  cost and lack of sophistication has limited the ability for
02800	such analysis, and the slow pace of automation has not  pressed  hard
02900	for analysis.  Now it will be possible to find common fabrication and
03000	assembly  operations  throughout  a  large   company,   to   discover
03100	commonality  in  design and components.  The success of that analysis
03200	depends upon representation  of  fabrication  operations  and  shape.
03300	Representation  is  a  current  concern in robotics labs and may have
03400	early applications. This orientation to the manufacturing  system  is
03500	already   beginning  to  have  effect  with  the  "group  technology"
03600	approach, which attempts to describe parts and operations, and  group
03700	them together in one part of the factory.  Until the establishment of
03800	such analysis and commonality of operations, automating will be  made
03900	much more difficult.
04000	
04100	Considerations are similar for programming automation devices such as
04200	the assembly station we describe.  Even if programming is  difficult,
04300	there  will  be  considerable  applications  for  dexterous  assembly
04400	systems. For example, for the military, it is preferable to stockpile
04500	programs than to stockpile parts or systems for contingency. In batch
04600	production, tapes can be saved to cut  stockpiling.  But  to  achieve
04700	full  effect  on  industry, robot systems must be easy to program for
04800	new tasks.  Our current work is concerned  with  strategy  generation
04900	and   automating   visual  feedback.  Time-sharing  systems  will  be
05000	necessary  to  perform  scheduling,  resource   allocation,   routing
05100	operations,  etc.  Thus  there is a natural division of labor between
05200	the time-sharing system and small machines. The strategy  generation,
05300	automating   feedback,  and  interactive  system  for  computer-aided
05400	mechanical design expect to reside in the central system.
05500	
05600	A major part of our effort is directed toward the representation of
05700	complex shapes, for use in vision and strategies.  While it might
05800	seem that this effort has only distant applications, for the reasons
05900	discussed above, early applications will be for convenient man/machine
06000	interaction for programming robots and for systematization. 
06100	We can expect that in general, much of the work in symbolic representation
06200	of the real world will have early payoff.
06300	Present
06400	non-intelligent robots are programmed by specifying points along a
06500	path to be repeated.  With computer control, optional trajectories
06600	can be executed in case of contingency.  The engineer must then specify
06700	what normal function is, how to monitor it, and how to recognize
06800	recoverable situations.  
06900	To do so, he can specify this symbolically in interaction with a program
07000	in which he specifies normal and abnormal conditions, shape of parts 
07100	(usually some variant of standard parts), the mating of parts for assembly,
07200	and schematic descriptions of operations for insertion and inspection.
07300	The significance of representations in general is to aid these processes.
07400	Our representation provides a natural way of speaking about the shape
07500	of objects, in terms of a structured description of part/whole with
07600	primitives which resemble many fabrication operations.  
07700	The better the representation, the more it
07800	provides intuition for programs as to the intent of operations
07900	and the function of parts.
08000	
08100	A representation aids communication in both directions.  We contend
08200	that our representation is important for computer graphics.  This
08300	allows display of the machines concepts to the user, and allows
08400	greater use of predictive mechanisms based on internal models.
08500	Input could be made visually with a depth measuring system such
08600	as we have been using for the last two years.  Since the operation would
08700	be done on a central facility, the computer would be adequate for this task.
08800	We expect that the representation will be of considerable use in monocular
08900	vision also.  We hope to substantiate these predictions.  Another application
09000	is automating design of numerically machined parts.  That will be simplified
09100	to the extent that the machine and the user speak the same language of
09200	subparts and their descriptions.  Benefits would come from ease of programming
09300	and interaction, variable tolerances for critical parts and better
09400	monitoring operations.
     

00100	
00200		  THE PILOT ASSEMBLY STATION
00300	
00400	We seek funds from other sponsors to apply robotics techniques to
00500	industrial assembly and inspection.  We will implement a prototype
00600	assembly station and a support system for programming the assembly
00700	station for its tasks.
00800	
00900	The  assembly  station is assumed to be normally connected to a large
01000	time-shared computer facility. Not only does this appear to be a good
01100	design  for  our  problem, but it is also necessary for an integrated
01200	programmed automation facility.  The only emphasized statement in the
01300	ARPA  study  report  [Anderson  1972]  is:   "The  maximum  impact of
01400	computers  in  manufacturing  will  come  from  complete,   real-time
01500	computer  cognizance  and  control  of  all  processes and resources,
01600	allowing the precise scheduling and allocation of these resources."
01700	
01800	Each assembly station will consist of a small digital  computer,  one
01900	or  more  manipulators  and cameras. Manipulators and cameras will be
02000	combined as modules in systems to suit a given task,  and  will  be
02100	digitally controlled by the station computer.
02200	
02300	The program in the station computer will be flexible as to the number
02400	and position of manipulators and cameras. It will control input  from
02500	the  cameras,  provide for some pre-processing of the picture and the
02600	application of simple tests to the visual data.     In  the  case  of
02700	manipulators  it will provide for simultaneous servo control. It will
02800	also process  other  inputs,  such  as  from  touch  sensors  on  the
02900	manipulators,  from photocell interrupt devices, limit switches, etc.
03000	Both analog and binary outputs will be available for the  control  of
03100	other devices.
03200	
03300	Each  station  computer will have access to a task program specifying
03400	the actions and tests it should perform to carry out its task.   This
03500	task program will be compiled by the central computer.    Compilation
03600	will consist of the analysis of predicted scenes in order to  specify
03700	simple tests that the station computer can carry out, the calculation
03800	of collision free trajectories for the manipulators, the sequence  of
03900	operations  and  simple  contingency plans. etc.  We are planning for
04000	tasks  whose  normal  execution  does  not  require  detailed   scene
04100	analysis.  If scene analysis becomes necessary  (e.g. a dropped part)
04200	we envision the station sending an image to the central computer  for
04300	processing.
04400	
04500	The  central  computer will perform overall control of the individual
04600	assembly stations and the relative rates of production.   If  at  any
04700	station  a  planned test fails, for which a contingency plan does not
04800	exist, or any unexpected sensor signals  are  received,  the  central
04900	computer will also be interrupted and appropriate actions taken.
05000	
05100	Most of the hardware required for the assembly station has been
05200	developed for  the  laboratory.  Manipulators  and  cameras  will  be
05300	discussed  in detail below. The major additional problem is to refine
05400	the  existing  designs  to  meet  the  demands   of   an   industrial
05500	environment.
05600	
05700	In  addition we will purchase a small computer such as the PDP-11 and
05800	interface the existing arms to it together with one or more  cameras.
05900	A  software  package for the small computer will then be developed in
06000	order to control the manipulators, read the TV cameras and to perform
06100	low  level  visual  analysis.  Programs  to plan tasks, calibrate the
06200	system and to perform higher level visual  analysis,  error  recovery
06300	and  strategy  planning will be written in a high level language for
06400	the time-sharing processor.
06500	
06600	The most difficult part of the assembly station, and the heart of the
06700	of the proposal is a demonstration of the feasibility of the station.
06800	We plan to do this by developing programs which actually assemble and
06900	inspect a variety of fairly complex manufactured items.
07000	
07100	A  typical  example is our current project, assembly of an automobile
07200	water pump. The major  parts  are  to  be  located,  the  screws  are
07300	supplied by a feeder, as is standard.  Locating the body and cover of
07400	the pump require a gross localization, then a  fine  localization  of
07500	screw  holes.    Touch  provides  a  final  accuracy  and independent
07600	coordination of the hand.   The  placing  of  alignment  pins  within
07700	screw  holes  could  be monitored visually; although in this case,the
07800	previous actions provide adequate accuracy.   Placing  of  the  cover
07900	over  alignment  pins  can  be  carried out without vision.  Improved
08000	force  sensing  now  under  development  will  be  useful  for   that
08100	operation.
08200	
08300	As  we move to more difficult tasks, the sophistication required goes
08400	up quite rapidly.  The  current  manipulation  routines  seem  to  be
08500	expandable  to cover anticipated problems, especially since we intend
08600	to employ special purpose manipulators (tools) for some  tasks.   The
08700	main part of our effort will be vision for assembly; an assembly task
08800	will be performed using vision, touch, and two hands.     The  vision
08900	effort  will  be  integrated with other abilities, and while no great
09000	generality will be stressed, new modules will  be  incorporated  into
09100	the  system,  and a set of techniques will evolve.  These will depend
09200	heavily on techniques similar to  those  of  our  recent  studies  in
09300	visual  feedback  and hand/eye coordination [Gill 1972]. The internal
09400	model of the world will be  used  to  predict  visual  appearance  of
09500	selected  areas  of  the  scene, and a set of verification procedures
09600	will confirm or contradict the predictions.  Many of  the  operations
09700	will  be  connected  with  supplying  feedback  to  manipulation, for
09800	alignment and insertion. These visual procedures will have models for
09900	the parts and approximate locations.
10000	
10100	
10200	
10300	
10400	This and other visual tasks connected with assembly will initially be
10500	This and other visual tasks connected with assembly will initially be
10600	performed as special case  problems.   In  addition,  to  make  these
10700	techniques   available,   we  will  pursue  development  of  strategy
10800	programs.  This step is essential if intelligent assembly devices are
10900	to  be conveniently programmed for complex tasks. Within the division
11000	of labor between large and small machines, this facility must  reside
11100	in  the  central  time-sharing  facility which has large memory, file
11200	storage, and problem-solving languages. The system must have a  model
11300	of  the  parts  for  assembly,and  a  sequence  of  instructions  for
11400	assembly. It will be necessary to have a representation of shape  and
11500	models for usual assembly operations such as insertion and alignment,
11600	and the ability to solve simple problems  in  connection  with  these
11700	operations,  all  this  to  enable  convenient communication with the
11800	user. When these facilities exist, the addition of new  features  and
11900	operations  can be incorporated smoothly, so that users can call on a
12000	growing library of planning modules.  The result of a planning effort
12100	is  a  cycle  of  machine  planning and man-machine interaction which
12200	results in a plan  network  for  the  mini  computer  which  controls
12300	individual machines.
     

00100	Will  the  incremental  cost  of  touch,  force and visual sensors be
00200	offset by the opening of new  capabilities,  and  will  the  cost  be
00300	offset  by  reduced  cost  of  system design for machines which allow
00400	loose  alignment  tolerances?     We  estimate  that  by  1977,   the
00500	incremental cost of visual sensors, a laser ranging device such as we
00600	use now, additional memory requirements, with the multiple of two  in
00700	marketing,  will  add about 15%-20% to current manipulator prices. If
00800	labor costs continue to rise, part of the increase  will  be  offset,
00900	but cost comparisons will not change appreciably over that time scale
01000	unless significant new mechanical designs are made.      A conclusion
01100	is that sensor-equipped robots will be available at slight additional
01200	cost.   We  explore  the  benefits  of  incremental  improvements  to
01300	manipulators.
01400	
01500	We do not expect mechanical production costs to drop as  dramatically
01600	as electronics costs fall with integration.   The most promising cost
01700	reduction development would be high-production standard modules  such
01800	
01900	as  robots.     As an example, consider the Stanford arm; produced in
02000	unit  quantity,  parts,  manufacturing   and   assembly   costs   are
02100	approximately  $10K.    The  arm  is much simpler than an automobile;
02200	there is no reason why it should  cost  more.
02300	
02400	Potentially,  a  robot with two arms, minicomputer control, and touch
02500	sensors can be constructed for about $10K in quantity.
02600	This  is  the  range of price of present transfer machines which have
02700	limited market because they have limited motion, no control, and  are
02800	thus  locked  out  of  the economy of scale. There is a dynamic which
02900	leads to economy of scale; the greater the versatility, the wider  is
03000	the  market for applications. This will be true particularly in small
03100	production and job shop  operations  which  have  been  difficult  to
03200	automate.     The larger the production, the more versatility for the
03300	same initial cost as special  devices.     Also,  with  high  volume,
03400	standardization of maintenance, and availability of parts and service
03500	are improved.  The quality of the device is also improved.
03600	
03700	The economies of scale in production of assembly modules are  matched
03800	by  economies  of  intellectual  scale  in  development of design for
03900	automation.   This approach to advanced automation saves much  wasted
04000	effort.   The  design process is modularized and easily communicable;
04100	there  is  a  common  discipline  which  can  be   widely   absorbed;
04200	developments  of  many  sources can be integrated.    Programming and
04300	training done once for one machine can be transferred to all machines
04400	in  a plant, and even transferred nationwide.   Functional modularity
04500	has been emphasized in our studies. Modularity  is  clearly  possible
04600	and there is a powerful economic incentive to assemble widely varying
04700	systems from one, two, or three arm modules, force and touch modules,
04800	vision  and  ranging  modules,  with planning and monitoring modules.
04900	This  would  also  entail  software  modularity  with  computer-aided
05000	mechanical  design  and  job  specification  for robots on a local or
05100	network time-sharing service.
05200	
05300	The alternative is special design for each operation.  We expect that
05400	at  some  point,  design  will be cheaper with standard robot modules
05500	than with special purpose designs, just as it is now more  economical
05600	to  use  mini-computer  controllers than special purpose controllers.
05700	Modularity has the  potential  to  reduce  design  time  and  improve
05800	predictability  of  design time and costs, permitting better planning
05900	and  response  to  competition.  To  some  extent,  the   dehumanized
06000	industrial  worker  now fills the role of standard module, so that we
06100	do not expect discontinuous change. The continuity  with  decades  of
06200	experience  with  design  for  human  versatility  in assembly can be
06300	relied upon;  robots  with  some  versatility,  sensing,  contingency
06400	actions,  and  dexterity  will  provide a transition to newer schemes
06500	which will evolve more slowly. Advances which  benefit  special  case
06600	automation  and traditional trends in automation for human production
06700	will also aid general purpose devices.   For example, new  techniques
06800	in  bonding and fastening will simplify many assembly tasks to within
06900	the  capability  of  robots,  while  still  too  complex  for  simple
07000	servomechanisms.  Another  advantage of general purpose techniques is
07100	the ease with which the size of the robot can be scaled up  or  down.
07200	
07300	           
07400	There  appears to be no difficulty in making very large or very small
07500	assembly stations which use essentially the same programs.
07600	
07700	Versatile  assembly  devices  have   an   efficiency   advantage   in
07800	utilization   in   medium   scale  and  job  shop  production;  their
07900	utilization efficiency can be higher  because  versatility  allows  a
08000	variety  of  operations  to  be  scheduled  at one station.       The
08100	availability of computers for production scheduling  will  allow  the
08200	flexibility  to  be used to advantage.  Robots would also have longer
08300	life over which to amortize costs,  since  they  could  operate  with
08400	greatly  changed  production operations, and evolve with new hardware
08500	and software modules.     They would have higher  reliability,  since
08600	they  would  have monitoring, contingency action, and error recovery.
08700	They would be self-aligning, self-calibrating, and  self-testing.  In
08800	many  cases,  system  costs  would  be smaller with advanced assembly
08900	devices.  With simple devices, transfer and assembly operations  must
09000	be  broken  down  into  small  operations  with  transfer  operations
09100	between.       This   introduces   extra   transfer   devices    with
09200	orientation-preserving   or   re-orienting   steps,  and  overlapping
09300	operations which can be combined in an advanced  device.  Robots  can
09400	cut  down on real estate, since several operations can be done at one
09500	station.  A robot which  can  pick  parts  from  a  cart  would  save
09600	considerable  space over belts and assembly lines which require large
09700	floor space.    Because of  the  adaptibility  of  work  stations,  a
09800	quicker  response to market and competition is possible.   Production
09900	runs can be smaller, with less  capital  tied  up  in  inventory  and
10000	unused  machines.   There can be better response to the needs of high
10100	technology  industry  which  often  requires  small   quantities   of
10200	instruments.    There  has  been  difficulty  in  meeting  the  tight
10300	tolerances necessary for operation of non-intelligent devices.     It
10400	is  hard  to  estimate the value of systems which work within current
10500	industrial tolerance, rather than requiring radical  improvements  in
10600	engineering practice, but that continuity does seem important.
10700	
10800	The  development  of  general  purpose  assembly devices will benefit
10900	manufacturers and a broad spectrum of industry,  particularly  medium
11000	scale  manufacturing.    Partially  because  these  techniques do not
11100	create a new product and because they have very wide  utility,  there
11200	does  not seem to be a private institution with the competence or the
11300	desire to undertake this program.   It is  an  appropriate  area  for
11400	governmental action to help meet a national need.
11500	
11600	But given sensor-equipped robots at little extra cost, they will have
11700	visual ability  to  sense  color,  strong  contrasts  in  brightness,
11800	recognize  known  objects  from two dimensional outlines (top view of
11900	flat object) or three dimensionally using depth, to a certain  extent
12000	they  will  be  able  to  recognize  three  dimensional objects using
12100	monocular recognition.   They will be able to visually  align  mating
12200	parts    in    assembly    operations,    increase   reliability   by
12300	self-calibration and flaw and disaster detection,  and  permit  large
12400	tolerances  in parts transfer, even permitting parts to be taken from
12500	
12600	a bin.       Several of these abilities are not yet available in  the
12700	laboratory,   but   with  known  conditions  and  objects,  and  some
12800	engineering (e.g. color coding) are achievable within the  time  span
12900	of a few years.
     

00100		RESEARCH IN VISION
00200	
00300	Our past work on visual feedback [Gill 1972] in stacking objects  and
00400	inserting  objects  in  holes  is  directly  relevant  to  vision for
00500	assembly.  The system provides feedback for picking up, then  at  the
00600	final  position,  with  the hand still holding the block and in view,
00700	predicts the appearance of edges and corners within  a  small  window
00800	around  the  faces  to  be  aligned, and uses a corner/edge finder to
00900	determine alignment error. Incremental motion capability of  the  arm
01000	is   used   for   error   correction.        That  work  incorporates
01100	self-calibration to coordinate hand and eye  coordinate  systems  and
01200	provide  for  reliability,  as  well as to limit search by accurately
01300	predicting expected features. Two well-developed systems exist to aid
01400	future  work  in  visual  feedback.   GEOMED  [Baumgart  1972a]  is a
01500	geometric editor which allows  the  input  of  geometric  models  and
01600	manipulation  of  these  models.  OCCULT [Baumgart 1972b] is a hidden
01700	line elimination program which gives a symbolic output, valuable  for
01800	vision, in a few seconds.
01900	
02000	We  have  a  set  of  visual operations which provide confirmation of
02100	predictions and direct  interaction  with  the  visual  image.    For
02200	outdoor  scenes,  color is an important basis for region forming, and
02300	for preliminary scene organization. Several preliminary color  region
02400	analysis  programs  have been developed [Binford, unpublished, Bajcsy
02500	1972].   These will be extended as described below. An edge  operator
02600	[Hueckel 1969,1972] and a region-based corner/edge finder [Gill 1972]
02700	both operate on small windows of the scene, as directed by  a  higher
02800	level.    An  edge  verifier  is  used  to  confirm  or  reject edges
02900	predicted from context [Tenenbaum  1970].  Color  identification  has
03000	long  been  demonstrated.  Recently,  a  program has been implemented
03100	based on Land's Retinex model of color  constancy  in  vision.   This
03200	involves  taking  intensity measurements with the camera through red,
03300	green, and blue filter, and setting up a calibration scheme  for  the
03400	Goethe  color  triangle in the absence of any external color standard
03500	(white).  This is achieved by normalizing the intensities of  regions
03600	within  the  scene through each filter, and defining a color triangle
03700	derived from this ordering data, with  the  center  of  the  triangle
03800	designated  as  white. The regions may then be assigne color names in
03900	the normal way.  The program as it stands can  cope  reasonably  well
04000	with  table  top  scenes under varied lighting conditions - sunlight,
04100	quartz-halogen and flourescent, and therefore shows the same kind  of
04200	color constancy as do humans.
04300	
04400	We  have  directed part of our work to analysis of outdoor scenes and
04500	to  techniques  which  are   especially   important   for   planetary
04600	exploration  and  the automatic driver.  Outdoor scenes were analyzed
04700	with a combination of  color  and  texture  region  analysis  [Bajcsy
04800	1972];  depth  cues  were  taken  from  the  gradient  of texture.  A
04900	symbolic representation of texture and of  the  world  were  adopted;
05000	descriptors of texture elements were obtained as analytic expressions
05100	from the Fourier spectrum.  A considerable improvement could be  made
05200	by  combining  these  techniques with several techniques which obtain
05300	depth  information  from  correlation  of  images.  Motion   parallax
05400	[Nevatia  draft]  has  been  used  with  a succession of images.  The
05500	successive images differ little from  one  another  and  allow  local
05600	correlation  without  ambiguity.   In  a  sense,  this  combines  the
05700	advantages of narrow and wide angle stereo, since stereo ambiguity is
05800	avoided, while large baselines can be accumulated by tracking regions
05900	over several images. Another system incorporates  several  facilities
06000	in  wide  angle  stereo  [Hannah, Quam]; local statistics are used to
06100	limit search for correspondence of small areas  in  two  images.   An
06200	interesting  feature  is  self-orientation:  given  two cameras whose
06300	orientation is unknow, the system uses guided search to  establish  a
06400	few  correspondences, then determines the relative orientation of the
06500	two cameras.  Such self-orientation and  self-calibration  facilities
06600	have  considerable  significance  for industrial systems which can be
06700	maintained  calibrated.   These  are  complementary   depth   sensing
06800	techniques;  motion  parallax  can  be  used  for  planning paths and
06900	characterizing large scale terrain features.  Two  camera  stereo  is
07000	important  for  local  manipulation  and  disaster avoidance.  Visual
07100	feedback with stereo vision is much more powerful than  without.   We
07200	have   a  program  which  has  demonstrated  stereo  visual  feedback
07300	[Binford, unpublished].
07400	
07500	A thesis by Agin [Agin 1972] showed considerable progress in
07600	
07700	representation and description of complex objects.  He implemented  a
07800	laser  ranging  device [Binford 1970] which was operational two years
07900	ago.   Using depth data and a representation of shape [Binford  1971]
08000	which  we  have developed, we have been able to describe the parts of
08100	objects  like  a  doll,  snake,  and   glove   [Agin   1972].     The
08200	representation is based on segmenting objects into parts connected at
08300	joints.   Parts  are  described  by  a  volume  representation   with
08400	arbitrary  cross  section defined along a space curve.  The data were
08500	analyzed by techniques suggested by the representation.   Preliminary
08600	segmentation  was made into parts suggested by some primitives of the
08700	representation, smoothly  varying,  locally  cylindrical  parts.   An
08800	iterative  procedure  defined  an  approximation  to  the axis of the
08900	segment, fit  circular  cross  sections  normal  to  the  axis,  then
09000	obtained  an  estimate  of  a global cross section function which was
09100	then used to extend the axis and cross section as far as  consistent.
09200	
09300	A study of visual and depth ranging techniques has been  carried  out
09400	which  provides  a background for choosing sensors and interface, and
09500	making cost estimates [Binford 1972,Earnest 1967].    An  interesting
09600	result  is  that  the  cost of a laser ranging system such as ours is
09700	less than $1k given a tv system. We have maintained a calibrated hand
09800	and  vision system for several years [Sobel 1970], and have developed
09900	self-calibration  techniques  with  one  and  two  eyes   [Yakimovsky
10000	unpublished, Hannah 1972, Gill 1972].
10100	
10200	A  system  for  understanding  scenes of blocks from single views has
10300	been implemented [Falk 1970]; it segments  objects,  matches  against
10400	internal  models,  predicts  appearance  from  its  assumptions,  and
10500	searches for verification and contradiction based  on  the  predicted
10600	image.    Support and spatial relations are also obtained.  There has
10700	been past work on programs  to  obtain  line  drawings  as  input  to
10800	recognition  systems.  An earlier system was based on limited context
10900	of expecting continuation of edges at vertices and predicting missing
11000	edges  in certain expected configurations [Grape 1970]. A more recent
11100	approach has been to use models within the process of generating line
11200	drawings,  so  that  the  most  reasonable next guess can be made and
11300	context used intrinsically in the formation of line drawings.
11400	
11500	VISUAL DEVELOPMENT FOR ASSEMBLY
11600	
11700	In the pump assembly, visual tasks consisted  of  gross  location  of
11800	large parts, then using the information to predict location of images
11900	of holes, and monitoring of insertion into holes.   These  tasks  are
12000	typical  of  the  simple visual operations which we will implement or
12100	adapt.   We will greatly expand our efforts with visual feedback.  At
12200	present, visual feedback techniques are limited to polyhera.  We will
12300	improve image prediction techniques to predict images of objects with
12400	
12500	curved edges.  We will develop a simple region analysis program which
12600	assumes adequate contrast to locate the curved edges predicted within
12700	a small window.   Color  will  be  used  when  necessary.    We  will
12800	implement  two-dimensional matching of boundary curves with predicted
12900	curves.  These will be combined with our existing edge  operator  and
13000	corner/line  operator.   We  will  maintain  a  calibrated system and
13100	improve calibration updating so that reliability is maintained  at  a
13200	high level.
13300	
13400	VISION RESEARCH
13500	
13600	We  will improve the laser hardware so that it can be used widely and
13700	extend our work with representation of complex  objects  using  depth
13800	data.   We  are  able  to make complete descriptions of objects which
13900	consist of a single primitive segment, such as a torus, a snake or  a
14000	cone.   The  programs  can  segment dolls, toy horses, in intuitively
14100	natural ways [Agin 1972]. It is necessary to improve our segmentation
14200	process  in  order to proceed to joining primitive segments at joints
14300	and make complete descriptions of  complex  objects.   When  we  have
14400	these  descriptions,  the computer will have symbolic internal models
14500	of objects.   No recognition has yet been done, although we are in  a
14600	position  to  recognize  those  objects which have only one primitive
14700	segment, for which we can make complete descriptions. For recognition
14800	of  more complex objects, once we have complete descriptions, we will
14900	encode the basic properties of the  representation  within  a  visual
15000	memory   scheme   which   allows   access   of  hypothesized  similar
15100	three-dimensional shapes, with  associated  surface  information,  to
15200	compare more closely with extensive verification.  We intend to apply
15300	large parts of our programs for description and recognition of curved
15400	objects  using  only  monocular intensity information. This is a much
15500	harder problem, but within our representation,  shares  much  of  the
15600	same  code,  and  relies  on  the  same  conceptual  basis  of  local
15700	three-dimensional models.
15800	
15900	
16000	We  will  continue  work  on  color region analysis which is based on
16100	proximity rather than contiguity.  This  was  the  plan  outlined  in
16200	previous  progress  reports, and has shown necessary by our work with
16300	textured  scenes  [Bajcsy  1972].  Another  effort  combines   region
16400	analysis  with  a decision theoretic approach based on a world model.
16500	We anticipate that these modules will aid the  integrated  system  to
16600	function in more complex environments than have been attempted in the
16700	past.  There is a growing effort on navigation of vehicles in outdoor
16800	scenes.   While  this  is  not  directly  relevant  to the industrial
16900	environment, it does  reflect  the  growing  involvement  in  complex
17000	visual  tasks,  and will lead to further development of an integrated
17100	visual system.
17200	
17300	A strategy project is underway dealing with the  blocks  world.   The
17400	project  begins  from  the  idea  that robot vision cannot be studied
17500	except as a part of intelligent behavior in general.  Visual  ability
17600	is  FOR  something,  and  without  reasonable  expectations about the
17700	world, the task of seeing is bound to  be  extremely  difficult.   We
17800	plan  to  implement  a  system which can do simple manipulation tasks
17900	("pick up the red block", "put all the  cubes  in  the  box")  in  an
18000	environment of plane-faced objects, and operating in a dialogue mode.
18100	The system will have a good but imperfect world model all  the  time,
18200	and the essence of its ability will be the constant use of this model
18300	to understand situations and in particular  to  recover  from  errors
18400	("Is  that  block  no longer there because the arm knocked it off?").
18500	The main point of the project is to  find  a  good  organization  for
18600	knowledge  about  the  simple world in question:  we expect that when
18700	this is done the  complex  subroutines  of  current  vision  programs
18800	(linefinders  etc)  will be found to be unnecessarily complex in that
18900	they embody in an inefficient way at a low level, knowledge which our
19000	system  will contain at a higher level. In part, this will be done by
19100	expanding the facilities available.  A  stereo  vision  effort  using
19200	line  drawings  and  integrating  stereo  with  the  segmentation and
19300	recognition processes.
19400	
19500	We are at work on a program which exploits new features of SAIL for
19600	dealing with multi-process tasks in a problem-solving environment.  The 
19700	aim is to construct a driver program to accomplish a simple assembly
19800	task under somewhat uncertain conditions.  Making use of the new language
19900	features, one can arrange for such a program to cope with errors in the
20000	execution of tasks - e.g. to recover from dropping things, breaking things,
20100	and further to try several different but possibly cooperative methods for
20200	solving a problem.  It is intended that this program will have its own vision procedures
20300	specially adapted for finding tools, e.g. wrenches, screwdrivers, etc.,
20400	and parts, e.g. nuts and bolts. An interpreter is being developed to allow
20500	flexible description of the task by the user. 
20600	
     

00100	 MANIPULATOR DEVELOPMENT
00200	
00300	Manipulator   research  began  at  Stanford  Artificial  Intelligence
00400	Laboratory  in  1965  with  the  acquisition  and  programming  of  a
00500	prototype  Rancho Orthotic Arm.  A later version of this arm was used
00600	in vision directed block stacking programs.
00700	
00800	Our present arm, the result of a continuing design project,  replaced
00900	the  Rancho Arm in 1970. This arm like its predecessor was controlled
01000	by a digital servo program running under the time  sharing  computer.
01100	The new arm was used in the "Instant Insanity" demonstration in 1971.
01200	A second arm of the same type,  with  only  minor  modifications,  is
01300	currently  being interfaced to the computer.   Both arms will share a
01400	common work space and will  be  able  to  be  used  individually,  or
01500	together for two handed tasks.
01600	
01700	Each  joint  of  the  arm  is calibrated in terms of torque v.  drive
01800	level and it is possible to predict the joint drive levels to produce
01900	a  given  force at the hand. Further, for the hand to exhibit degrees
02000	of freedom of motion, various joints are selected to be "free",  that
02100	is  to have no position feedback applied to them. By exerting a force
02200	at the hand and providing that certain degrees of freedom exist it is
02300	possible  to  perform  a  much  wider  class  of  tasks  than  simple
02400	positional control, for instance screwing in  screws,  mating  parts,
02500	inserting tools and turning cranks.
02600	
02700	Touch  sensors on the fingers allow the hand to sense objects without
02800	disturbing them.   When activated the arm will stop when any  of  the
02900	sensors  touch.  Branching arm instructions can test the state of the
03000	sensors and modify the execution of an arm program. When  screwing  a
03100	nut  onto  a  bolt, for instance, the hand is first centered over the
03200	bolt by touching it.
03300	
03400	Control of the arms is by a series of high level instructions;  these
03500	instructions  may  be executed directly, or may be assembled together
03600	into an arm program. Arm programs may then be executed,  as  desired,
03700	by  the  control  program.   The control instructions may be accessed
03800	directly by other programs, using the message procedure mechanism, or
03900	in the form of symbolic source file.
04000	
04100	Positional  accuracy  at  the  hand  is  typically  0.1  inches  with
04200	repeatibility of 0.02 inches. Force control at the hand is within  10
04300	ounces. The data rate to run the arm from an assembled arm program is
04400	of the order of 100 bits/second.
04500	
04600	Manipulator research moves  ahead  simultaneously  on  three  fronts:
04700	control,  primitives  and  devices.      Tasks are translated into an
04800	existing hand language and where the task cannot be  accomplished  in
04900	terms  of  existing  primitives,  new  primitives are formulated in a
05000	consistent manner.   When a new primative is  formulated,  real  time
05100	routines  are  developed  to perform the action.  This also leads, in
05200	many cases, to the development of  new  hardware.       Also  as  new
05300	
05400	                                 10                                  developments  occur  in  hardware design, existing real time routines
05500	are either re-written or modified.
05600	
05700	Research is now concentrating on the following areas:
05800	
05900	Strain gauge instrumentation of the wrist, in order to allow for more
06000	sensitive  interaction  of  the  hand with a task.   High resolution,
06100	high sensitivity touch sensors, to increase tactile  recognition  and
06200	control  abilities.     These  developments  will lead in turn to new
06300	control concepts.
06400	
06500	Development of tools for the hand. These tools  can  be  instrumented
06600	such  that  the  computer can feel and or see at the end of the tool.
06700	This should be a very important area of research as  it  goes  beyond
06800	present human capabilities.
06900	
07000	Two handed arm control, the planning and execution of tasks requiring
07100	the coordinated motion of both arms. This will require  an  extension
07200	of  the  existing  servo  program and the analysis of synchronization
07300	problems. The development of collision avoiders, so that arms will
07400	not collide with other objects and with each other.
07500	
07600	Modification  of  the  existing  arm  design  to  provide  for better
07700	incremental motions, more degrees of freedom and a more rugged design
07800	suitable  for  industrial  use.   Development of arm and eye computer
07900	interfaces.   These contain  synchronized  analog/digital  converters
08000	for input and current controlled drivers for output.
08100	
08200	The  development  of  new hands is expected as more complex tasks are
08300	attempted.   At present, with only one hand, we have been  performing
08400	the  positioning  type  of  hand  actions.   When  the  second arm is
08500	operational we will be able to perform holding and pulling actions as
08600	well.
08700	
08800		D.  SYSTEM REQUIREMENTS FOR ADVANCED AUTOMATION
08900	
09000	A  key  element in the development of any large programming effort is
09100	the software support.  There has been continual  effort  along  these
09200	lines  at  the  Stanford  Artificial Intelligence Laboratory and many
09300	useful ideas have been developed.  We propose to extend the  research
09400	along  two  separate lines:  support for research and system software
09500	for operating automated systems.  We will briefly describe  our  past
09600	efforts in this area and what we propose to undertake.
09700	
09800	The Stanford Artificial  Intelligence  Laboratory  has  pioneered  in
09900	several  areas  of system programming, including timesharing systems,
10000	interactive graphics, and programming languages.
10100	
10200	A great deal of additional effort has gone  into  direct  support  of
10300	hand-eye  research.   Very  early,  we  established a set of standard
10400	formats and library routines [Pingle 72a].  A  programming  language,
10500	SAIL,  was  implemented  in  1968.  The language incorporates special
10600	
10700	                                 11                                  features for large real-world-oriented problem solving systems and is
10800	currently  in  use  at  a number of Artificial Intelligence and other
10900	laboratories.
11000	
11100	As  we began to assemble larger and more sophisticated collections of
11200	programs, we felt a need  for  flexible  mechanisms  for  cooperating
11300	sequential  processes.  The resulting monitor and language features [
11400	Feldman & Sproull 1971]  have  worked  well  for  us  and  have  been
11500	influential   on   other  projects.    Recently,  we  have  added  an
11600	additional set of features to SAIL  to  facilitate  non-deterministic
11700	and  event-directed  programming.   The early reactions to this work,
11800	both within the laboratory and without has been very encouraging.
11900	
12000	We expect relatively little work to  be  needed  on  further  systems
12100	support  for  basic  research.   There will undoubtedly be efforts to
12200	take advantage of new hardware and of new programming ideas that  may
12300	develop  in  the  course  of  this program.   Almost all of this work
12400	should be covered by other funding.
12500	
12600	The development of software for the  assembly  station  will  require
12700	more  of  our  effort.  The octupus configuration requires code to be
12800	resident in the local computer and in the shared machine.    The  low
12900	level  programs in the local machine will not be modified often, thus
13000	we can assume that a PDP-l0  resident  compiler  will  suffice.   The
13100	planning programs and general vision programs will be resident in the
13200	shared system.  There are two possible configurations for the  shared
13300	computer  system:   A PDP-10 equivalent machine or a software package
13400	in ASA Standard Fortran IV which will run on any  sufficiently  large  2nd ???
13500	machine.    One of the tasks under this proposal would be to evaluate
13600	these two alternatives.
13700	
13800	A major share of the programming research effort will be concentrated
13900	on developing a programming system for automated assembly.  This will
14000	combine ideas  from  our  past  work  on  languages,  representation,
14100	computer  graphics and arm control.  We view this task as a branch of
14200	the larger effort on automatic programming being undertaken  in  this
14300	laboratory aad elsewhere.
14400	
14500	
14600	
14700	
14800	
14900	
15000	
15100